Papers with semantic parsers
Compositional generalization with a broad-coverage semantic parser (2022.starsem-1)
Copied to clipboard
| Challenge: | Recent work has shown that compositional generalization on COGS is difficult and complex. |
| Approach: | They propose a compositional semantic parser that solves compositional generalization on COGS dataset. |
| Outcome: | The AM parser solves compositional generalization on the COGS dataset. |
Neural Semantic Parsing (P18-5)
Copied to clipboard
| Challenge: | Semantic parsing is the study of translating natural language utterances into machine-executable programs. |
| Approach: | They will describe the various approaches researchers have taken to translate natural language into a formal language . they will also discuss why much recent work has chosen to use standard programming languages instead of more linguistically-motivated representations. |
| Outcome: | This paper will describe the various approaches researchers have taken to translate natural language into a formal language. |
Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention (D19-61)
Copied to clipboard
| Challenge: | Semantic parsers are used to convert user’s natural language commands to executable logical form in intelligent personal agents. Labeled datasets required to train such parser are expensive to collect, and are never comprehensive. |
| Approach: | They propose to use a sequence-to-sequence/tree attention based attention-based sequence-based parsers which support fast near real time retraining. |
| Outcome: | The proposed parsers can maintain high accuracy and fast retraining time while leveraging paraphrases already present in the training dataset. |
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques (2023.acl-short)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs)2 have been adopted for semantic parsing due to their promising performance and straightforward architectures. |
| Approach: | They propose to use token preprocessing to preserve semantic boundaries of tokens produced by LM tokenizers and special tokens to mark the boundaries of aligned components. |
| Outcome: | The proposed techniques improve the performance of pre-trained language models on two text-to-SQL semantic parsing datasets. |
AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate semantic parsers that answer questions on databases require large amounts of annotated data. |
| Approach: | They propose a method to generate semantic parsers that answer questions on databases . they use automatic paraphrasing and template-based parsing to find alternative expressions . |
| Outcome: | The proposed method achieves 69.8% answer accuracy on natural questions, 16.4% higher than state-of-the-art models and 5.2% lower than the same model trained with human data. |
Meta-Learning for Domain Generalization in Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to parsing use standard supervised learning, but little attention has been given to domain generalization. |
| Approach: | They propose a meta-learning framework which targets zero-shot domain generalization for semantic parsing. |
| Outcome: | The proposed framework significantly boosts parser performance on English and Chinese spider datasets. |
Content Differences in Syntactic and Semantic Representation (N19-1)
Copied to clipboard
| Challenge: | Syntactic analysis plays an important role in semantic parsing, but the nature of this role remains a topic of ongoing debate. |
| Approach: | They propose to use Universal Dependencies and UCCA as test cases to compare syntactic and semantic schemes. |
| Outcome: | The proposed comparison methodology can be used for fine-grained evaluation of UCCA parsing, highlighting both challenges and potential sources for improvement. |
Detecting Compositionally Out-of-Distribution Examples in Semantic Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural network models suffer from performance losses when faced with compositionally out-of-distribution data. |
| Approach: | They propose to use neural semantic parsers to detect compositionally out-of-distribution (OOD) data. |
| Outcome: | The proposed methods perform well on the standard SCAN and CFQ datasets. |
The Web as a Knowledge-Base for Answering Complex Questions (N18-1)
Copied to clipboard
| Challenge: | Recent work on reading comprehension made headway in answering simple questions, but tackling complex questions is still an ongoing research challenge. |
| Approach: | They propose to decompose complex questions into a sequence of simple questions and compute the final answer from the sequence of answers. |
| Outcome: | The proposed framework improves performance from 20.8 precision@1 to 27.5 precision@1. |
Prediction or Comparison: Toward Interpretable Qualitative Reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | Qualitative relationships are a significant portion of textual knowledge . current approaches use semantic parsers to transform natural language inputs into logical expressions or a "black-box" model to solve them in one step. |
| Approach: | They propose to use neural network modules to simulate qualitative reasoning tasks . they use two qualitative reasoning question answering datasets to test their methods . |
| Outcome: | Experiments on two qualitative reasoning question answering datasets show the proposed methods are general and general and interpretable. |
Semi-Supervised Lexicon Learning for Wide-Coverage Semantic Parsing (C18-1)
Copied to clipboard
| Challenge: | Semantic parsers rely on accurate and high-coverage lexicons, but they often use annotated logical forms to learn the lexic. |
| Approach: | They propose a semi-supervised learning framework that makes use of large text corpora and lexical resources. |
| Outcome: | The proposed framework improves on two benchmarks: Webquestions and Free917. |
On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex (2023.eacl-main)
Copied to clipboard
Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li
| Challenge: | Existing techniques for parsing natural-language utterances are vulnerable to adversarial attacks, requiring large amounts of labelled data and expensive human annotation. |
| Approach: | They propose to enhance the adversarial robustness of a prompt-based semantic parser based on a language model trained on code by constructing a set of demonstration examples. |
| Outcome: | The proposed method can be enhanced without significant amounts of labelled data or expensive human annotations on in-domain semantic parsing data. |
Text-to-SQL Error Correction with Language Models of Code (2023.acl-short)
Copied to clipboard
Ziru Chen, Shijie Chen, Michael White, Raymond Mooney, Ali Payani, Jayanth Srinivasa, Yu Su, Huan Sun
| Challenge: | Existing semantic parsers are not accurate enough for use in text-to-SQL parsing tasks. |
| Approach: | They propose to build clause-level edit models to correct SQL queries instead of token-level ones. |
| Outcome: | The proposed model improves the exact set match accuracy of different parsers by 2.4-6.5 and obtains up to 4.3 point absolute improvement over two strong baselines. |
Searching for Better Database Queries in the Outputs of Semantic Parsers (2023.findings-eacl)
Copied to clipboard
| Challenge: | generating a database query from a question in natural language is a longstanding task . the task is amplified when the system needs to generalize to databases unseen at training. |
| Approach: | They propose to generalize a query to databases unseen at training . they use state-of-the-art semantic parsers to find queries that meet the criterion . |
| Outcome: | The proposed approach finds that many queries pass all tests on different datasets. |
Weakly Supervised Semantic Parsing with Abstract Examples (P18-1)
Copied to clipboard
| Challenge: | training semantic parsers from weak supervision complicates training in two ways . spurious programs that accidentally lead to a correct denotation add noise to training . |
| Approach: | They propose to use tokens in both language utterance and program to map denotations to executable programs. |
| Outcome: | The proposed method improves performance and reaches 82.5% accuracy compared to the best reported accuracy so far. |
Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback (P18-1)
Copied to clipboard
| Challenge: | a recent study shows that counterfactual learning from human bandit feedback can improve neural semantic parsers . cost and difficulty of manually preparing large amounts of parses is a bottleneck for supervised learning . |
| Approach: | They propose to use human bandit feedback to apply counterfactual learning to neural parsing . they devise an easy-to-use interface to collect human feedback on semantic parses . |
| Outcome: | The proposed framework improves semantic parsers by reducing the cost of manual parsing . the proposed framework is based on human bandit feedback collected by the user . |
Semantic Parsing for Conversational Question Answering over Knowledge Graphs (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent years have seen an increasing number of applications aiming to build conversational interfaces based on information retrieval and user recommendation. |
| Approach: | They develop a dataset where user questions are annotated with Sparql parses and system answers correspond to execution results thereof. |
| Outcome: | The proposed parsers can be used to ground questions into queries over definitions in a knowledge graph with large vocabularies. |
Decoupling Structure and Lexicon for Zero-Shot Semantic Parsing (D18-1)
Copied to clipboard
| Challenge: | Existing methods for training semantic parsers in new domains require expensive supervision and lack the ability to generalize to new domain. |
| Approach: | They propose a zero-shot approach to parsing utterances in unseen domains . they map an utterant to an abstract, domain independent, logical form and replace slots with KB constants based on lexical alignment scores and global inference . |
| Outcome: | The proposed model achieves 53.4% accuracy on 7 domains in the OVERNIGHT dataset, significantly better than other zero-shot baselines and performs as good as a parser trained on over 30% of the target domain examples. |
Learning to Learn Semantic Parsers from Natural Language Supervision (D18-1)
Copied to clipboard
| Challenge: | Existing logical forms require a user to be familiar with the underlying structure to learn a semantic parser. |
| Approach: | They propose a method for training semantic parsers from natural language feedback . they use natural language inputs to parse feedback to leverage it as a form of supervision . |
| Outcome: | The proposed algorithm learns a semantic parser from users’ corrections expressed in natural language. |
Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention (2021.naacl-main)
Copied to clipboard
Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas
| Challenge: | Existing approaches to compositional generalization in semantic parsers focus on word-level alignments, but they focus on spans. |
| Approach: | They propose a span-level supervised attention loss that improves compositional generalization in semantic parsers by focusing on spans. |
| Outcome: | The proposed method improves on three benchmarks of compositional generalization. |
Iterative Search for Weakly Supervised Semantic Parsing (N19-1)
Copied to clipboard
| Challenge: | Recent work has focused on training semantic parsers via weak supervision from denotations alone. |
| Approach: | They propose an iterative training algorithm that alternates between searching for consistent logical forms and maximizing the marginal likelihood of the retrieved ones. |
| Outcome: | The proposed algorithm outperforms the previous best systems on WikiTableQuestions and Cornell Natural Language Visual Reasoning (NLVR) iteratively train models that provide guidance to subsequent models to search for logical forms of increasing complexity, thus dealing with spuriousness. |
Leveraging Code to Improve In-Context Learning for Semantic Parsing (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is an attractive approach for semantic parsing, but learning to parse to rare domain-specific languages from a few demonstrations is challenging. |
| Approach: | They propose to use Python instead of DSLs to augment prompts with a structured domain description. |
| Outcome: | The proposed approach improves accuracy and generalization across three datasets. |
On Robustness of Neural Semantic Parsers (2021.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing maps natural language (NL) utterances into logical forms (LFs) adversarial examples are created by adding tiny perturbations to inputs but can severely deteriorate model performance. |
| Approach: | They propose to construct robustness test sets based on existing benchmark corpora and to evaluate the effect of data augmentation. |
| Outcome: | The proposed method measures the performance of the proposed parsers on robustness test sets and evaluates the effect of data augmentation. |
Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for continual learning for semantic parsing fail to account for special properties of structured outputs . retraining from scratch is not feasible due to the fast growing number of tasks . |
| Approach: | They propose a continual learning method that uses sequential learning to learn tasks without accessing full training data from previous tasks. |
| Outcome: | The proposed method achieves a 3-6 times speedup compared to re-training from scratch. |
A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to build effective semantic parsers for Wizard-of-Oz are insufficient. |
| Approach: | They propose a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations. |
| Outcome: | The proposed model can predict precise dialogue states in WOZ conversations. |
Transparent Semantic Parsing with Universal Dependencies Using Graph Transformations (2022.coling-1)
Copied to clipboard
| Challenge: | Existing semantic parsers are based on deep learning, but rule-based approaches offer advantages . a drawback of neural semantic parses is that their output lacks explainability . |
| Approach: | They propose a method that maps a syntactic dependency tree to a formal meaning representation using a series of graph transformations. |
| Outcome: | The proposed method outperforms neural parsers in English, German, Italian and Dutch. |
Global Reasoning over Database Structures for Text-to-SQL Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing semantic parsers only select a set of database constants at training time . current models only consider local information, not global ones . |
| Approach: | They propose a semantic parser that globally reasons about the structure of the query to make a more contextually-informed selection of database constants. |
| Outcome: | The proposed model increases accuracy from 39.4% to 47.4% on a zero-shot semantic parsing dataset with complex databases. |
XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work focuses on English datasets, and it is unclear whether large language models can serve as competitive semantic parsers for other languages. |
| Approach: | They propose a framework that learns to retrieve relevant English exemplars for a given query to construct prompts. |
| Outcome: | The proposed framework learns to retrieve relevant English exemplars for a given query to construct prompts. |
Cross-lingual Text-to-SQL Semantic Parsing with Representation Mixup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that Rex can benefit from cross-lingual training and improve the effectiveness of semantic parsers. |
| Approach: | They propose a Representation Mixup Framework for effectively exploiting translations in the cross-lingual Text-to-SQL task. |
| Outcome: | The proposed framework can benefit from cross-lingual training and improve the effectiveness of semantic parsers, achieving state-of-the-art performance. |
TaPas: Weakly Supervised Table Parsing via Pre-training (2020.acl-main)
Copied to clipboard
| Challenge: | Answering natural language questions over tables is often seen as a semantic parsing task. |
| Approach: | They propose an approach to question answering over tables without generating logical forms by selecting table cells and optionally applying a corresponding aggregation operator. |
| Outcome: | The proposed approach outperforms or rivals existing models on three different datasets and performs on par with the state-of-the-art on WikiSQL and WikiTQ. |
Dscorer: A Fast Evaluation Metric for Discourse Representation Structure Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | Discourse representation structures (DRSs) are scoped semantic representations for texts of arbitrary length. |
| Approach: | They propose a new metric which converts box-style DRSs to graphs and measures the overlap of n-grams. |
| Outcome: | Experiments show that Dscorer computes accuracy scores that are correlated with Counter at a fraction of the time. |
Weakly Supervised Semantic Parsing with Execution-based Spurious Program Filtering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train a semantic parser from weak supervision focus on exploiting similarities between examples based on domain-specific knowledge. |
| Approach: | They propose a domain-agnostic filtering mechanism based on program execution results to identify and filter out programs with significantly different semantics from the other programs. |
| Outcome: | The proposed method improves the performance of existing weakly-supervised parsers by incorporating a majority vote on the program search results. |
Compositional Semantic Parsing across Graphbanks (P19-1)
Copied to clipboard
| Challenge: | Existing semantic parsers that map sentences to graph-based meaning representations are hand-designed for specific graphbanks. |
| Approach: | They propose a compositional neural semantic parser which achieves competitive accuracies across graphbanks. |
| Outcome: | The proposed system achieves competitive accuracies across a variety of graphbanks. |
Laziness Is a Virtue When It Comes to Compositionality in Neural Semantic Parsing (2023.acl-long)
Copied to clipboard
Maxwell Crouse, Pavan Kapanipathi, Subhajit Chaudhury, Tahira Naseem, Ramon Fernandez Astudillo, Achille Fokoue, Tim Klinger
| Challenge: | Compositional generalization is a key feature of human intelligence and has been identified as a major point of weakness in neural methods for semantic parsing. |
| Approach: | They propose a neural parsing generation method that constructs logical forms from the bottom up, beginning from the logical form’s leaves. |
| Outcome: | The proposed method outperforms general-purpose parsers on a CFQ dataset and two other Text-to-SQL datasets while also being competitive with parser that have been tailored to each task. |
WikiBank: Using Wikidata to Improve Multilingual Frame-Semantic Parsing (2020.lrec-1)
Copied to clipboard
| Challenge: | Frame-semantic annotations exist for a tiny fraction of the world’s languages, however, Wikidata provides a common, distant supervision signal for semantic parsers. |
| Approach: | They propose a multilingual resource with partial semantic dependency structures that can be used to extend pre-existing resources rather than creating new man-made resources from scratch. |
| Outcome: | The proposed resource can be used to augment pre-existing resources or reduce the annotation effort for low-resource languages. |
Learning to Generate Programs for Table Fact Verification via Structure-Aware Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for table fact verification do not study generating latent programs from statements . current weakly supervised methods are limited due to huge search space with lots of spurious programs. |
| Approach: | They propose a structure-aware approach to do table fact verification by structure-based parsing . they leverage lexical features and structure features to generate program from statements . |
| Outcome: | The proposed method generates programs more accurately than existing parsers and achieves comparable performance to the SOTA on the large-scale benchmark TABFACT. |
Learning Programmatic Idioms for Scalable Semantic Parsing (D19-1)
Copied to clipboard
| Challenge: | In state-of-the-art semantic parsers map natural language instructions to source code . idioms improve the accuracy of semantic parses, allowing for faster decoding . |
| Approach: | They propose an iterative method to extract code idioms from large source code corpora . they use most-frequent subtrees of their syntax trees to train semantic parsers to apply them . |
| Outcome: | The proposed method improves the state-of-the-art semantic parsers' accuracy and training time by more than 50%. |
From Parse-Execute to Parse-Execute-Refine: Improving Semantic Parser for Complex Question Answering over Knowledge Base (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for parsing knowledge-base questions into executable logical forms have not been successful on complex KBQA. |
| Approach: | They propose a new semantic parser called KoPL to model the reasoning processes . they propose 'parse-execute-refine' paradigm to unlock reasoning ability . |
| Outcome: | The proposed parser performs better than the state-of-the-art on complex KBQA . the proposed parsed-execute-refine paradigm can model complex reasoning steps . |
Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that automatic generation of synthetic utterance-program pairs can alleviate the first problem, but its potential for the second has thus far been under-explored. |
| Approach: | They propose to generate synthetic utterance-program pairs for improving compositional generalization in semantic parsing by using structurally-diverse examples. |
| Outcome: | The proposed approach leads to dramatic improvements in compositional generalization and moderate improvements in the traditional i.i.d setup. |
ZeLa: Advancing Zero-Shot Multilingual Semantic Parsing with Large Language Models and Chain-of-Thought Strategies (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to augment multilingual datasets with labeled English data are lacking in annotated data. |
| Approach: | They propose a framework to augment English data and then use it to train parsers . they propose to use multilingual chain-of-thought prompting techniques to augment other languages' data . |
| Outcome: | The proposed framework augments English data in other languages and trains them with no demonstration samples in target languages. |